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REMARKS 

Claims 1-4, 6-12, and 14-40 are pending in this application. By this Amendment, 
claims 1, 3-4, 6-7, 9, 11-12, 14-15, 17, 21-22, 27, 32, 37-40 are amended and claims 5 and 13 
are canceled without prejudice or disclaimer of the subject matter contained therein. 

The courtesies extended to Applicant's representative by Examiner Siangchin and 
Supervisory Patent Examiner Au at the interview held September 1 , 2004, are appreciated. 
The reasons presented at the interview as warranting favorable action are incorporated into 
the remarks below and constitute Applicant's record of the interview. 

I. Information Disclosure Statement (IDS) 

References submitted with an IDS filed on September 13, 2001, have not been 
acknowledged by the Examiner. Accordingly, acknowledgment of the IDS by initializing the 
submitted PTO-1449 Form and forwarding a copy of the initialized PTO-1449 Form in the 
next Patent Office communication is respectfully requested. 

II. Obiection to the Drawings 

The Office Action objects to the drawings based on formal matters. Applicant 
respectfully submits that the Formal Drawings submitted with a Letter to the Official 
Draftsperson on September 14, 2001 obviate the objection based on formal matters. Further, 
the Office Action asserts that Fig. 10 does not correlate to the discussion found in the last 
paragraph of page 23 of the disclosure. Applicant respectfully submit the Fig. 10 represents a 
slit pattem of the board 606, which has been newly coded. It is respectfully submitted that the 
above-mentioned paragraph is clear on this matter. Accordingly, withdrawal of the objection 
to the drawings is respectfully requested. 
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III. Objection to the Specification 

1. Objection to the Title 

The Office Action objects to the title. The title has been amended to obviate the 
objection. 

2. Objection to the Disclosure 

The Office Action objects to the written description based on formal matters. The 
written description has been amended to obviate the objection. 

Regarding item d., as mentioned above, the slit pattem of the board 606 is re-coded as 
in Fig. 10 to distinguish over the slit pattem of the wall 605. It is respectfully submitted that 
this is clear in the discussion found in the last paragraph of page 23 of the disclosure. 

3. Objection to the Summary of the Invention 

The Office Action objects to the Summary of the Invention. The Summary of the 
Invention has been amended to obviate the objection. 

IV. The Claims Satisfy Formal Matters 

The Office Action objects to claims 6 and 14 based on formal matters. The claims 
have been amended to obviate the objection. 

V. Rejection Based on 35 U.S.C. $112^ Second Paragraph 

The Office Action rejects claims 7, 15, 17, 22, 27, 32 and 37-40 under 35 U.S.C. 
§112, second paragraph, as being infinite. The above-mentioned claims have been amended 
to overcome the rejection. 

VI. The Claims Define Patentable Subject Matter 

The Office Action rejects claims 1, 8, 9 and 16 under 35 U.S.C. §102(b) over Kang 
("A Multibaseline Stereo System with Active Illumination and Real-Time Lnage 
Acquisition'*); rejects claims 2-4, 7, 10-12 and 15 under 35 U.S.C. § 103(a) over Kang in view 
of BatUe ("Recent Progress in Coded Stmctured Light as a Technique to Solve the 
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Correspondence Problem: A Survey", Pattern Recognition, 1998); rejects claims 6 and 14 
under 35 U.S.C. § 103(a) over Kang in view of Mack (U.S. Patent No. 6,377,700); rejects 
claims 5 and 13 under 35 U.S.C. §103(a) over Kang in view of Perona (U.S. Patent No. 6, 
044, 165); rejects claims 27-30 under 35 U.S.C. §103(a) over Saund (U.S. Patent No. 
5,764,383) in view of Perona; rejects claims 17-24, 26-27, 31-34 and 36-40 under 35 U.S.C. 
§103(a) over Saund in view of Stolfo (U.S. Patent No. 5,668,897); and rejects claims 25 and 
35 under 35 U.S.C. § 103(a) over Saund in view of Stolfo, and further in view of Wellner 
(U.S. Patent No. 5,5 1 1 ,148). Applicant respectfully traverses the rejection. 

In particular, none of the applied references disclose or suggest an image processing 
part that retrieves only differential data between successive frame data in the time-series as 
storage data based on a result of the comparison of the frame data comparison part of the 
initial frame data and frame data subsequently transformed in the time-series, as recited in 
independent claim 1 ; retrieving only differential data between successive geometric- 
transformed intensity images in the time series as storage data based on a result of the 
comparison of the comparison step of the initial geometric-transformed intensity image and 
geometric-transformed intensity images subsequently transformed in the time-series; as 
recited in independent claim 9; an image processing part that retrieves only differential data 
between successive geometric-transformed intensity images in the time-series as storage data 
based on a result of the comparison of the image extracting part of the initial geometric- 
transformed intensity image and geometric-transformed intensity images subsequently 
transformed in the time-series, wherein the stored geometric-transformed intensity image is 
the initial geometric-transformed intensity image and the differential data between successive 
geometric-transformed intensity images in the time-series , as recited in independent claims 17 
and 38; and retrieving only differential data between successive geometric-transformed 
intensity images in the time-series as storage data based on a result of the comparison step of 
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the initial geometric-transformed intensity image and geometric-transformed intensity images 
subsequently transformed in the time-series, wherein the stored geometric-transformed 
intensity image is the initial geometric-transformed intensity image and the differential data 
between successive geometric-transformed intensity images in the time-series , as recited in 
independent claims 27, 37, 39 and 40. 

Kang discloses in the abstract a parallel depth recovery scheme which uses a light 
pattern with frequency modulated sinusoidally of varying intensity onto a target. 

BatUe discloses in the abstract coded structured light methods that are employed to get 
3D information. 

Mack discloses at col. 5, lines 49-51, using a non- visible light source to create 
structured light, 

Perona discloses at col. 2, line 66 - col. 3, line 8, a system for tracking, for example, a 
pen tip movement relative to a writing surface, by first determining an initial position of a pen 
tip. Then the image neighborhood around the initial position of the pen tip is assembled into 
a "kernel." This kemel is used to determine the position of the pen in subsequent frames. 
Importantly, Perona discloses at col. 3, lines 43-45 that Perona*s system preferably monitors 
relative movement of the writing implement, instead of imaging previously- written 
characters . 

Saund discloses at col. 5, lines 48-53, a method of using structured light to detect a 
three-dimensional object of a page, and thereby correct distortions caused by image 
perspective, skew, and compression caused by compression or elongation introduced by 
uneven surface of a bound document. 

Stolfo discloses at col. 11, lines 18-28, a system that stores a set of codebook of data 
images of financial instruments or standardized documents; scanning an instrument; 
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comparing the scanned image against the codebook; subtracting the codebook image from the 
scanned image; and compressing the leftover handwritten or post-printed information. 

Wellner discloses in Figs. 1-3 and on page 88, a DigitalDesk where a computer 
display is projected onto a desk, and video cameras pointed down at the desk feed an image- 
processing system that senses what a user is doing. 

However, none of the above references disclose or suggest an image processing part 
that retrieves only differential data between successive frame data in the time-series as storage 
data based on a result of the comparison of the frame data comparison part of the initial frame 
data and frame data subsequently transformed in the time-series, as recited in independent 
claim 1 ; retrieving only differential data between successive geometric-transformed intensity 
images in the time series as storage data based on a result of the comparison of the 
comparison step of the initial geometric-transformed intensity image and geometric- 
transformed intensity images subsequently transformed in the time-series; as recited in 
independent claim 9; an image processing part that retrieves only differential data between 
successive geometric-transformed intensity images in the time-series as storage data based on 
a result of the comparison of the image extracting part of the initial geometric-transformed 
intensity image and geometric-transformed intensity images subsequently transformed in the 
time-series, wherein the stored geometric-transformed intensity image is the initial geometric- 
transformed intensity image and the differential data between successive geometric- 
transformed intensity images in the time-series , as recited in independent claims 17 and 38; 
and retrieving only differential data between successive geometric-transformed intensity 
images in the time-series as storage data based on a result of the comparison step of the initial 
geometric-transformed intensity image and geometric-transformed intensity images 
subsequently transformed in the time-series, wherein the stored geometric-transformed 
intensity image is the initial geometric-transformed intensity image and the differential data 
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between successive geometric-transformed intensity images in the time-series , as recited in 
independent claims 27, 37, 39 and 40. 

Therefore, independent claims 1, 9, 17, 27 and 37-40 define patentable subject matter. 
Claims 2-4, 6-8, 10-12, and 14-16, 18-26 and 28-36 depend on the respective independent 
claims, and therefore also define patentable subject matter, as well as for the other features 
they recite. Accordingly, withdraw of the rejections under 35 U.S.C. § 102(b) and 35 U.S.C. 
§ 103(a) are respectfully requested. 
VII. Conclusion 

In view of the foregoing, it is respectfiiUy submitted that this application is in 
condition for allowance. Favorable reconsideration and prompt allowance of claims 1-4, 6- 
12, and 14-40 are earnestly solicited. 

Should the Examiner believe that anything fiirther would be desirable in order to place 
this application in even better condition for allowance, the Examiner is invited to contact the 
undersigned at the telephone number set forth below. 



Jamd^^/^Uff 
Registration No 



Respectfully sul 




27,075 



itted. 



Yong S. Choi 
Registration No 



43,324 



JAO:YSC/eks 



Date: September 7, 2004 



OLIFF & BERRIDGE, PLC 

P,0, Box 19928 
Alexandria, Virginia 22320 
Telephone: (703) 836-6400 



DEPOSIT ACCOUNT USE 

AUTHORIZATION 
Please grant any extension 

necessary for entry; 
Charge any fee due to our 
Deposit Account No. 15-0461 
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Amendments to the Specification 

Please replace the title as follows: 
IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND 
STORAGE MEDIUM FOR PICKED UP IMAGES 

Please cancel paragraphs at page 3, line 12 through page 13, line 26. 

Please add the following new paragraph after the subtitle "SUMMARY OF THE 
INVENTION" on page 3, Hne 1 1 : 

According to an aspect of the present invention, an image processing apparatus 
efficiently identifies, for example, input characters and the like by an intensity image analysis 
based on range information. The image processing apparatus performs distance measurement 
by performing three-dimensional measurement by pattem projection, for example, to written 
characters and gets an intensity image as a so-called actual image and an image for distance 
measurement in parallel, and identifies, for example, input characters by an intensity image 
analysis based on range information. 

The image processing apparatus, for example, picks up characters, pattems, and the 

like written on paper or like with a pen, performs geometric transformation for picked up 
images, and performs natural input character analysis and read processing. Furthermore, by 
comparing images picked up in a time-series, noise elimination and manuscript position 
modifications become possible. 

These and other characteristics and advantages of the present invention will become 

apparent by more detailed descriptions based on embodiments of the present invention to be 
described and accompanying drawings. 

Please add the following new paragraphs after the subtitle "DESCRIPTION OF THE 
PREFERRED EMBODIMENTS" on page 17, line 7: 
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The present invention has been made in view of the above circumstances and provides 

an image processing apparatus and an image processing method that perform distance 
measurement with a simple construction, gets an intensity image used as a so-called actual 
image and an image for distance measurement in parallel, and identifies input characters by 
an intensity image analysis based on range information. 

According to an aspect of the present invention, the image processing apparatus has: a 

three-dimensional image pickup part that includes a projecting part that projects patterns, a 
first image pickup part that picks up an intensity image and a projection pattern image fi-om 
the direction of an optical axis of the projecting part, and a second image pickup part that 
picks up the projection pattern image fi'om a direction different fi-om the optical axis of the 
projecting part. First range information is created based on a pattern picked up by the second 
image pickup part and a geometric transformation part that performs geometric 
transformation for the intensity image picked up by the first image pickup part based on the 
range information. 

Furthermore, in the image processing apparatus of the present invention, for areas 

where the amount of change of the picked-up pattern by the first image pickup part with 
respect to the projection pattern is equal to or greater than a predetermined value, new codes 
corresponding to the picked-up pattern by the first image pickup part may be assigned, and 
the first range information may be created fi"om the picked-up pattern by the second image 
pickup part, based on the new codes. 

Furthermore^ the image processing apparatus may fiirther have: a fi-ame data 

comparison part that makes comparisons between firame data images picked up in a time- 
series by the three-dimensional image pickup part; and an image processing part that 
eliminates noise data fi-om the fi-ame data images based on a result of making comparisons 
between the fi:'ame data images in the fi-ame data comparison part. 
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Furthermore, the image processing apparatus may further have: a frame data 

comparison part that makes comparisons between frame data images picked up in a time- 
series by the three-dimensional image pickup part: and an image processing part that modifies 
image positions of the frame data images based on a result of making comparisons between 
the frame data images in the frame data comparison part. 

Furthermore, the image processing apparatus may further have: a storage part that 

stores, as initial frame data, an initial image of frame data picked up in a time-series by the 
three-dimensional image pickup part; a frame data comparison part that makes comparisons 
between frame data images picked up in a time-series by the three-dimensional image pickup 
part: and an image processing part that extracts only differential data as storage data, based on 
a result of comparing, in the frame data comparison part, the initial frame data with frame 
data subsequently picked up. 

Furthermore, in the image processing apparatus of the present invention, the 

projecting part may have a light source to emit light of an invisible region: and the first and 
second image pickup parts may have a filter for transmitting light of an invisible region and a 
filter for cutting off light of an invisible region, and may pick up pattern projection images 
and intensity images in parallel. 

Furthermore, in the image processing apparatus of the present invention, for areas 

where the amount of change of the picked-up pattem by the first image pickup part with 
respect to the projection pattern by the projecting part is less than a predetermined value, 
second range information may be created by bringing the areas into correspondences with 
respective pieces of intensity information obtained by the first and second image pickup parts. 

Furthermore, in the image processing apparatus of the present invention, the second 

image pickup part may include plural image pickup parts that pick up the measurement target 
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at different angles, and range information may be created based on projection patterns 
respectively picked up by the plural image pickup parts. 

Furthermore, according to another aspect of the present invention, the image 

processing method includes: a projecting step that projects patterns by a projecting part; an 
image pickup step that picks up an intensity image and a projection pattem image by a first 
image pickup part from an optical axis direction of the projecting part, and picks up the 
projection pattem image by a second image pickup part from a direction different from the 
optical axis direction of the projecting part; a range information creation step that creates first 
range information, based on a picked-up pattem picked up by the second image pickup part; 
and a geometric transformation step that performs geometric transformation for the intensity 
image produced by the first image pickup part, based on the range information. 

Furthermore, in the image processing method of the present invention, the range 

information creation step may include the step of: for areas where the amount of change of 
the picked-up pattem by the first image pickup part with respect to the projection pattem is 
equal to or greater than a predetermined value, assigning new codes corresponding to the 
picked-up pattem by the first image pickup part, and creating the first range information fro m 
the picked-up pattem by the second image pickup part, based on the new codes. 

Furthermore, the image processing method may further include: a frame data 

comparison step that makes comparisons between frame data images picked up in a time- 
series in the image pickup step; and an image processing step that eliminates noise data from 
the frame data images, based on a result of making comparisons between the frame data 
images in the frame data comparison step. 

Furthermore, the image processing method may further include: the frame data 

comparison step that makes comparisons between frame data images picked up in a time- 
series in the image pickup step; and an image processing step that modifies image positions of 
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the frame data images, based on a result of making comparisons between the frame data 
images in the frame data comparison step. 

Furthemiore, the image processing method may further include: a storage step that 

stores an initial image of frame data picked up in a time-series in the image pickup step in a 
storage part as initial frame data; a frame data comparison step that makes comparisons 
between frame data images picked up in a time-series in the image pickup step: and an image 
processing step that extracts only differential data as storage data, based on a result of 
comparing, in the frame data comparison step, the initial frame data and frame data got 
subsequently. 

Furthermore, in the image processing method of the present invention, the pattern 

projecting step may use the light source as an invisible-region light source using infrared or 
ultraviolet light, and may form pattern light by invisible-region light: and the image pickup 
step may pick up pattern projection images and intensity images in parallel. 

Furthermore, in the image processing method of the present invention, for areas where 

the amount of change of the picked-up pattern by the first image pickup part with respect to 
the projection pattern by the projecting part is less than a predetermined value, second range 
information may be created by bringing the areas into correspondences with respective pieces 
of intensity information obtained by the first and second image pickup parts. 

Furthermore, in the image processing method of the present invention, the second 

image pickup part may include plural image pickup parts that pick up the measurement target 
at different angles, and the method may include the step of creating range information, based 
on projection patterns respectively picked up by the plural mage pickup parts. 

According to another aspect of the present invention, the image processing apparatus 

includes: a projecting part that projects light to an image holding medium to form an image 
thereon: an image pickup part that picks up the image formed on the image holding medium: 
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an intensity image acquisition part that acquires an intensity image, based on the image 
picked up in the image pickup part; a range information acquisition part that acquires range 
information from the picked-up image: a geometric transformation part that performs 
geometric transformation for the intensity image, based on the range information acquired in 
the range information acquisition part; and an image extracting part that extracts differences 
between the geometric-transformed intensity image and the intensity image acquired in 
adyance. 

Furthermore, in the image processing apparatus of the present inyention, the image 

holding medium is one of a manuscript sheet, whiteboard, blackboard, screen, walU and 
screen projection sheet. 

Furthermore, in the image processing apparatus of the present invention, the intensity 
image acquired in advance as a processing target in the image extracting part may be a 
preceding frame image inputted precedent to the geometric transformation part. 

Furthermore, in the image processing apparatus of the present invention, the intensity 

image acquired in advance as a processing target in the image extracting part may be image 
data stored in advance in the storage part. 

Furthermore, the image processing apparatus may include: a document database in 

which plural pieces of document format data are stored; and a document identifvdng part that 
performs matching between the geometric-transformed intensity image and format images 
stored in the document database. The image extracting part may extract differences between 
the geometric-transformed intensity image and format images stored in the document 
database. 

Furthermore, the image processing apparatus may further have a character 

transformation processing construction that reads character data extracted by the image 
extracting part and transforms it to character data replaceable as code values. 
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Furthermore, the image processing apparatus may further include: an authentication 

information database in which handwriting history data of registered users is stored; and an 
authentication processing part that inputs the geometric-transformed intensity image and 
performs matching between the input image and handwriting history data stored in the 
authentication information database. 

Furthermore, in the image processing apparatus of the present invention, the 

authentication information database may store handwriting history data and signature shape 
data of registered users: and the authentication processing part may input the geometric- 
transformed intensity image, and performs matching between the input image and the 
handwriting history data stored in the authentication information database, and between the 
input image and the signature shape data. 

Furthermore, the image processing apparatus may further include a display part that 

displays an image produced as a result of performing geometric transformation for the 
intensity image, based on the range information in the geometric transformation part. 

Furthermore, in the image processing apparatus of the present invention, a distance 

between an image holding medium and the image pickup part may be fixed: the image 
processing apparatus may include a storage part that stores range information acquired by the 
range information acquisition part; and the geometric transformation part performs geometric 
transformation for the intensity image, based on the range information stored in the storage 
part. 

Furthermore, according to another aspect of the present invention, the image 

processing method includes: a projecting step that projects light to an image holding medium 
to form an image thereon; an image pickup step that picks up the image formed on the image 
holding medium; an intensity image acquisition step that acquires an intensity image, based 
on the image picked up in the image pickup step; a range information acquisition step that 
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* 

acquires range information from the picked-up image; a geometric transformation step that 
performs geometric transformation for the intensity image, based on the range information 
acquired in the range information acquisition step; and an image extracting step that extracts 
differences between the geometric-transformed intensity image and the intensity image 
acquired in advance. 

Furthermore, in the image processing method of the present invention, the image 

holding medium is one of a manuscript sheet, whiteboard, blackboard, screen, wall, and 
screen projection sheet. 

Furthermore, in the image processing method of the present invention, the intensity 

image acquired in advance as a processing target in the image extracting step may be a 
preceding frame image inputted precedent to the geometric transformation step. 

Furthermore, in the image processing method of the present invention, the intensity 

image acquired in advance as a processing target in the image extracting step may be image 
data stored in advance in the storage part. 

Furthermore, the method may further include: a document identifying step that 

performs matching between the geometric-transformed intensity image and format images 
stored in the document database in which plural pieces of document format data are stored. 
The image extracting step may extract differences between the geometric-transformed 
intensity image and format images stored in the document database. 

Furthermore, the method may flirther have a character transformation processing step 

that reads character data extracted in the image extracting step and transforms it to character 
data replaceable as code values. 

Furthermore, the method may include: an authentication processing step that inputs 

the geometric-transformed intensity image and performs matching between the input image 
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and handwriting history data stored in the authentication information database in which 
handwriting history data of registered users is stored. 

Furthermore, in the image processing apparatus of the present invention, the 

authentication information database may store handwriting history data and signature shape 
data of registered users; and the authentication processing step may input the geometric- 
transformed intensity image, and performs matching between the input image and the 
handwriting history data stored in the authentication information database, and between the 
input image and the signature shape data. 

Furthermore, the method may further include a display step that displays an image 

produced as a result of performing geometric transformation for the intensity image, based on 
the range information in the geometric transformation step. 

Furthermore, in the image processing method of the present invention, a distance 

between an image holding medium and the image pickup part may be fixed: the image 
processing method may include a storage step that stores range information acquired by the 
range information acquisition part; and the geometric transformation step performs geometric 
transformation for the intensity image, based on the range information stored in the storage 
step. 

Furthermore, according to another aspect of the present invention, a storage medium 

readable by a computer, which stores a program of instructions executable by the computer to 
perform method steps for performing image processing is provided. The method includes the 
steps of: projecting light to an image holding medium to form an image thereon; picking up 
the image formed on the image holding medium; acquiring an intensity image based on the 
image picked up in the image pickup step; acquiring range information from the picked-up 
image: performing geometric transformation for the intensity image based on the range 
information acquired in the range information acquisition step; and extracting difference 
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between the geometric-transformed intensity image and the intensity image acquired in 
advance. 

Furthermore, according to another aspect of the present invention, an image 

processing apparatus includes: a projecting part that projects Ught; an image pickup part that 
picks up the projected light: an intensity image acquisition part that acquires an intensity 
image from the picked-up light; a range information acquisition part that acquires range 
information from the picked-up light; a geometric transformation part that performs 
geometric transformation for the intensity image, based on the range information; and an 
image extracting part that extracts differences between the intensity image subjected to the 
geometric transformation and an intensity image acquired in advance. 

Furthermore, according to another aspect of the present invention, an image 

processing method includes: a projecting step that projects light; an image pickup step that 
picks up the projected light; an intensity image acquisition step that acquires an intensity 
image from the picked-up light; a range information acquisition step that acquires range 
information from the picked-up light; a geometric transformation step that performs 
geometric transformation for the intensity image, based on the range information; and an 
image extracting step that extracts differences between the intensity image subjected to the 
geometric transformation and an intensity image acquired in advance. 

Furthermore, according to another aspect of the present invention, a storage medium 

readable by a computer, which stores a program of instructions executable by the computer to 
perform method steps for performing image processing is provided. The method includes the 
steps of: a projecting step that projects light; an image pickup step that picks up the projected 
light; an intensity image acquisition step that acquires an intensity image from the picked-up 
light; a range information acquisition step that acquires range information from the picked-up 
light; a geometric transformation step that performs geometric transformation for the intensity 
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image, based on the range information: and an image extracting step that extracts differences 
between the intensity image subjected to the geometric transformation and an intensity image 
acquired in adyance. 

By providing a program in a computer-readable format, processing is performed on a 

compute system according to the program. By installing the computer program in the 
computer system, cooperative operations are deliyered on the computer system and the same 
effect as other aspects of the present invention can be obtained. 

Please replace the paragraph beginning on page 21, line 22, with the following 
rewritten paragraph: 

As the light sources 106 shown in FIG. 2, light sources (whether visible region or 
invisible region) of wavelength band in which image pickup is possible may be used, ha this 
case, as the camera 3 (shown as 103 in the figure), a CCD camera of progressive scan type is 
used, and the camera 1 (shown as 101 in the figure) and the camera 2 (shown as 102 in the 
figure) are not limited to a specific configuration. However, it is desirable to use a CCD 
camera of the same configuration in terms of match with the camera 3 (103). Patterns are 
projected fi:-om the light sources 106 and the three cameras 1 to 3 (101 to 103) perform image 
pickup at the same time. The cameras obtain light passing through the filters 504302 and 
50^304 (see FIG. 3) by the image pickup apparatuses 303 and 305, thereby obtaining images 
at a time. 

Please replace the paragraph beginning on page 22, line 21, with the following 
rewritten paragraph: 

FIG. 4 is a flowchart for re-coding. Each slit pattern is longitudinally split for each 
slit width (step lOOl Sll ) to create square cells. An average value of strength is taken for 
each of the created cells, and the average value is defined as the strength of the cell (step 
1002 S12 ). Sequentially fi-om the center of an image, the strengths of corresponding cells 
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between a projection pattern and a pick-up pattern etre compared, and it is judged whether a 
difference between the strengths of the cells is equal to or greater than a threshold value 
because the patterns change due to factors such as the reflectivity of an object and a distance 
from the object (step 1003 S13V Unless the difference is equal to or greater than a threshold 
value, re-coding for all picked-up cells terminates (step 1007 S17 ). 

Please replace the paragraph beginning on page 23, line 5, with the following 
rewritten paragraph: 

If the difference is equal to or greater than a threshold value, it is judged whether the 
cell is a cell of new strength (step 100 4 S14) . When the cell is a cell of new strength, a new 
code is created and allocated (step 1005 S15 ). When the cell is not a cell of new strength, a 
code is created using the placement of slit pattems identifiable with parts of other occurrences 
(step 4^06816). This terminates re-coding (step 4W7S17). 

Please replace the paragraph beginning on page 41, line 16, with the following 
rewritten paragraph: 

By forming the document identifying part 2001 and the document database 2002 
described in the second embodiment within the personal computer (PC) 2803, data written by 
a user onto a document format projected on the whiteboard 2801 using, e.g., an OHP 
2805 2804 can be subjected to the same processing as described in the second embodiment 
example. 
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Amendments to the Claims : 

The following listing of claims will replace all prior versions, and listings, of claims in 
the application: 

1. (Currently Amended) An image processing apparatus, comprising: 

a three-dimensional image pickup part that includes a projecting part that projects a 
pattem, a first image pickup part that picks up an intensity image and a projection pattern 
image from the direction of an optical axis of the projecting part, and a second image pickup 
part that picks up the projection pattem image from a direction different from the optical axis 
of the projecting part, the three-dimensional image pickup part creating first range 
information based on a pattem picked up by the second image pickup part; and 

a geometric transformation part that performs geometric transformation for the 
intensity image picked up by the first image pickup part, based on the first range 
information, information; 

a storage part that stores, as initial frame data, an initial image of frame data in a time- 
series transformed by the geometric transformation part: 

a frame data comparison part that makes comparison between successive frame data 

images in the time-series transformed by the geometric transformation part: and 

an image processing part that retrieves only differential data between successive frame 

data in the time-series as storage data based on a result of the comparison of the frame data 
comparison part of the initial frame data and frame data subsequently transformed in the time- 
series. 

2. (Original) The image processing apparatus according to claim 1, wherein, for an 
area where the amount of change of the pattem picked up by the first image pickup part with 
respect to the projection pattem is equal to or greater than a predetermined value, new code 
corresponding to the pattem picked up by the first image pickup part is assigned, and the first 
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range information is created from the pattern picked up by the second image pickup part 
based on the new code. 

3. (Currently Amended) The image processing apparatus according to claim 1, 
furthor comprisingi wherein 

a frame data comparison part that malcoG comparison bot^^^oen frame data imag e s 
pick e d up in a tim e s e ri e s by the thr ee dim e nsional imago pickup part; and 

mtthe image processing part tha^eliminates noise data from the frame data image 
based on a result of the comparison between the frame data images in the frame data 
comparison part. 

4. (Currently Amended) The image processing apparatus according to claim 1, 
furth e r comprisingi wherein 

a fram e data comparison part that malc e s comparison b e tw ee n fram e data imag e s 
pick e d up in a tim e s e ri e s by th e thr ee dim e nsional imag e piclcup part; and 

»the image processing part that-modifies a position of the frame data image based on 
a result of the comparison between the frame data images in the frame data comparison part. 

5. (Canceled). 

6. (Currently Amended) The image processing apparatus according to claim 1, 
wherein: 

the projecting part has a light source to emit light of an invisible region of a 
wavelength band : and 

the first and second image pickup parts have a filter for transmitting light of the 
invisible region of the wavelength band and a filter for cutting off Hght of- m the invisible 
region of the wavelength band , and pick up the projection pattern image and intensity image 
in parallel. 
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7. (Currently Amended) The image processing apparatus according to claim 1, 
wherein, for an area where the amount of change of the pattem picked up by the first image 
pickup part with respect to the projection pattem by the projecting part is less than a 
predetermined value, second range information is created by bringing the ar e a into deriving a 
correspondence wit bbetween intensity information obtained by the first and second image 
pickup parts. 

8. (Original) The image processing apparatus according to claim 1, wherein the 
second image pickup part includes plural image pickup parts that pick up the measurement 
target at different angles, and range information is created based on projection patterns 
respectively picked up by the plural image pickup parts of the second image pickup part. 

9. (Currently Amended) An image processing method, comprising: 
projecting a pattem by a projecting part; 

picking up an intensity image and a projection pattem image by a first image pickup 
part fi-om an optical axis direction of the projecting part, and picking up the projection pattem 
image by a second image pickup part from a direction different from the optical axis direction 
of the projecting part; 

creating first range information based on the pattem picked up by the second image 
pickup part; mid 

performing geometric transformation for the intensity image produced by the first 
image pickup part based on the range information. information; 

storing an initial geometric-transformed intensity image in a time-series transformed 

in the geometric transformation step: 

making comparison between successive geometric-transformed intensity images in the 

time-series transformed in geometric transformation step; and 
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retrieving only differential data between successive geometric-transformed intensity 

images in the time series as storage data based on a result of the comparison of the 
comparison step of the initial geometric-transformed intensity image and geometric- 
transformed intensity images subsequently transformed in the time-series. 

10. (Original) The image processing method according to claim 9, wherein the range 
information creation step includes the step of: 

for an area where the amount of change of the pattern picked up by the first image 
pickup part with respect to the projection pattern is equal to or greater than a predetermined 
value, assigning new code corresponding to the pattern picked up by the first image pickup 
part, and creating the first range information from the pattem picked up by the second image 
pickup part based on the new code. 

11 . (Currently Amended) The image processing method according to claim 9, further 
comprising: 

malcing comparison b e tw ee n fram e data imag e s pick e d up in a tim e s e ri e s in th e 
imag e pickup st e p; and 

eliminating noise data from the frame data image based on a result of the comparison 
between the frame data images in the frame data comparison step. 

12. (Currently Amended) The image processing method according to claim 9, ftirther 
comprising: 

making comparison b e tw ee n fram e data imag e s pick e d up in a tim e s e ri e s in th e 
imago pickup step; and 

modifying a position of the frame data image based on a result of the comparison 
between the frame data images in the frame data comparison step. 

13. (Canceled) 
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14. (Currently Amended) The image processing method according to claim 9, 
wherein: 

a pattem light is formed by a invisible r e gion Hght of an invisible-region of a 
wavelength band by using an invisibl e r e gion light sourc e using infrared or ultraviolet light as 
the light source; and 

the pattem projection image and intensity image are picked up in parallel. 

15. (Currently Amended) The image processing method according to claim 9, further 
comprising the step of: 

for an area where the amount of change of the pattem picked up by the first image 
pickup part with respect to the projection pattern by the projecting part is less than a 
predetermined value, creating second range information by bringing the aroa into deriving a 
correspondence wttb between intensity information obtained by the first and second image 
pickup parts. 

16. (Original) The image processing method according to claim 9, wherein: 
the second image pickup part includes plural image pickup parts that pick up the 

measurement target at different angles, and includes the step of creating range information 
based on projection pattems respectively picked up by the plural image pickup parts of the 
second image pickup part. 

17. (Currently Amended) An image processing apparatus, comprising: 

a projecting part that projects light to an image holding medium to form an image 
thereon; 

an image pickup part that picks up the image on the image holding medium projected 
by the projecting part; 

an intensity image acquisition part that acquires an intensity image based on the image 
picked up by the image pickup part; 
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a range information acquisition part that acquires range information from the picked- 
up image; 

a geometric transformation part that performs geometric transformation for the 
intensity image based on the range information acquired in the range information acquisition 

an image extracting part that extracts difference between a geometric-transformed 
intensity image and thean intensity image acquired in advanc e . advance; 

a storage part that stores, as the geometric-transformed intensity image, an initial 

geometric-transformed intensity image in a time-series transformed by the geometric 
transformation part; 

the image extracting part making comparison between successive geometric- 
transformed intensity images in the time-series transformed by the geometric transformation 
part; and 

an image processing part that retrieves only differential data between successive 

geometric-transformed intensity images in the time-series as storage data based on a result of 
the comparison of the image extracting part of the initial geometric-transformed intensity 
image and geometric-transformed intensity images subsequently transformed in the time- 
series, wherein the stored geometric-transformed intensity image is the initial geometric- 
transformed intensity image and the differential data between successive geometric- 
transformed intensity images in the time-series. 

18. (Original) The image processing apparatus according to claim 17, wherein the 
image holding medium is one of a manuscript sheet, whiteboard, blackboard, screen, wall, 
and screen projection sheet. 
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19. (Original) The image processing apparatus according to claim 17, wherein the 
intensity image acquired in advance as a processing target in the image extracting part is a 
preceding frame image inputted precedent to the geometric transformation part. 

20. (Original) The image processing apparatus according to claim 1 7, further 
comprising: 

a storage part that stores image data, 
wherein the intensity image acquired in advance as a processing target in the image extracting 
part is the image data stored in advance in the storage part. 

21. (Currently Amended) The image processing apparatus according to claim 17, 
further comprising: 

a document database in which plural pieces of document format data are stored; and 
a document identifying part that performs matching between the geometric- 
transformed intensity image and the pieces of document format data stored in the document 
database, 

wherein the image extracting part extracts differences between the geometric- 
transformed intensity image and the pieces of document format data stored in the document 
database. 

22. (Currently Amended) The image processing apparatus according to claim 17, 
further comprising a character transformation processing part that reads character data 
extracted by the image extracting part and transforms it to charact e r data r e plac e abl e as a codo 
value identifies the character data with identification data . 

23. (Original) The image processing apparatus according to claim 17, further 
comprising: 

an authentication information database in which handwriting history data of registered 
users is stored; and 
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an authentication processing part that inputs the geometric-transformed intensity 
image and performs matching between the input image and handwriting history data stored in 
the authentication information database. 

24. (Original) The image processing apparatus according to claim 23, wherein: 

the authentication information database stores handwriting history data and signature 
shape data of registered users; and 

the authentication processing part inputs the geometric-transformed intensity image 
and performs matching between the input image and the handwriting history data stored in the 
authentication information database, and between the input image and the signature shape 
data. 

25. (Original) The image processing method according to claim 17, further 
comprising a display part that displays an image produced as a result of performing geometric 
transformation for the intensity image, based on the range information in the geometric 
transformation part. 

26. (Original) The image processing apparatus claim 17, further comprising: 

a storage part that stores range information acquired by the range information acquisition part, 
wherein a distance between the image holding medium and the image pickup part is fixed, 
and the geometric transformation part performs geometric transformation for the intensity 
image based on the range information stored in the storage part. 

27. (Currently Amended) An image processing method, comprising: 
projecting light to an image holding medium to form an image thereon; 
picking up the image projected on the image holding medium; 

acquiring an intensity image based on the image picked up in the image pickup step; 
acquiring range information from the picked-up image; 
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performing geometric transformation for the intensity image based on the range 
information acquired in the range information acquisition step; and 

extracting difference between the geometric-transformed intensity image and tbean 
intensity image acquired in ad vanc e . advance ; 

storing, as the geometric-transformed intensity image, an initial geometric- 
transformed intensity image in a time-series transformed in the geometric transformation step; 

making comparison between successive geometric-transformed intensity images in the 

time-series transformed in the geometric transformation step: and 

retrieving only differential data between successive geometric-transformed intensity 

images in the time-series as storage data based on a result of the comparison step of the initial 
geometric-transformed intensity image and geometric-transformed intensity images 
subsequently transformed in the time-series, wherein the stored geometric-transformed 
intensity image is the initial geometric-transformed intensity image and the differential data 
between successive geometric-transformed intensity images in the time-series. 

28. (Original) The image processing method according to claim 27, wherein the 
image holding medium is one of a manuscript sheet, whiteboard, blackboard, screen, wall and 
screen projection sheet. 

29. (Original) The image processing method according to claim 27, wherein the 
intensity image acquired in advance as a processing target in the image extracting step is a 
preceding frame image inputted precedent to the geometric transformation step. 

30. (Original) The image processing method according to claim 27, wherein the 
intensity image acquired in advance as a processing target in the image extracting step is 
image data stored in advance in a storage part. 

31. (Original) The image processing method according to claim 27, further 
comprising: 
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storing plural pieces of document format data in a document database; and 
performing matching between a geometric-transformed intensity image and the pieces 
of document format data stored in the document database, 

wherein the image extracting step extracts difference between the geometric- 
transformed intensity image and the pieces of document format data stored in the document 
database. 

32. (Currently Amended) The image processing method according to claim 27, 
further comprising: 

reading character data extracted in the image extracting step and transforming it to 
charact e r data r e plac e abl e as a cod e valu e identifies the character data with identification data . 

33. (Original) The image processing method according to claim 27, further 
comprising: 

storing handwriting history data of registered users in a authentication information 
database; and 

inputting the geometric-transformed intensity image and performing matching 
between the input image and the handwriting history data stored in the authentication 
information database. 

34. (Original) The image processing apparatus according to claim 33, further 
comprising: 

in addition to the handwriting history data, storing signature shape data of registered 
users in the authentication information database; and 

inputting the geometric-transformed intensity image and performing matching 
between the input image and the handwriting history data stored in the authentication 
information database, and between the input image and the signature shape data. 
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35. (Original) The image processing method according to claim 27, further 
comprising: 

displaying an image produced as a result of performing geometric transformation for 
the intensity image based on the range information. 

36. (Original) The image processing method according to claim 27, further 
comprising: 

storing range information acquired in the range information acquiring step, 
wherein a distance between an image holding medium and the image pickup part is fixed and 
the geometric transformation for the intensity image is performed based on the range 
information stored in the storage step. 

37. (Currently Amended) A storage medium readable by a computer, the storage 
medium storing a program of instructions executable by the computer to perform method 
steps for performing image processing, the method comprising the steps of: 

projecting light to an image holding medium to form an image thereon; 
picking up the image formed on the image holding medium; 

acquiring an intensity image based on the image picked up in the image pickup step; 
acquiring range information from the picked-up image; 

performing geometric transformation for the intensity image based on the range 
information acquired in the range information acquisition step; 
and 

extracting difference between the geometric-transformed intensity image and thean 
intensity image acquired in advance. advance: 

storing, as the geometric-transformed intensity image, an initial geometric- 
transformed intensity image in a time-series transformed in the geometric transformation step; 
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making comparison between successive geometric-transformed intensity images in the 

time-series transformed in the geometric transformation step; and 

retrieving only differential data between successive geometric-transformed intensity 

images in the time-series as storage data based on a result of the comparison step of the initial 
geometric-transformed intensity image and geometric-transformed intensity images 
subsequently transformed in the time-series, wherein the stored geometric-transformed 
intensity image is the initial geometric-transformed intensity image and the differential data 
between successive geometric- trans formed intensity images in the time-series. 

38. (Currently Amended) An image processing apparatus, comprising: 

a projecting part that projects light; 

an image pickup part that picks up the projected light; 

an intensity image acquisition part that acquires an intensity image from the picked-up 

light; 

a range information acquisition part that acquires range information from the picked- 
up light; 

a geometric transformation part that performs geometric transformation for the 
intensity image based on the range information; and 

an image extracting part that extracts difference between the geometric-transformed 
intensity image and thean intensity image acquired in advanc e . advance: 

a storage part that stores, as the geometric- trans formed intensity image, an initial 

geometric-transformed intensity image in a time-series transformed by the geometric 
transformation part: 

the image extracting part making comparison between successive geometric- 
transformed intensity images in the time-series transformed by the geometric transformation 
part; and 
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an image processing part that retrieves only differential data between successive 

geometric-transformed intensity images in the time series as storage data based on a result of 
the comparison of the image extracting part of the initial geometric-transformed intensity 
image and geometric-transformed intensity images subsequently transformed in the time- 
series, wherein the stored geometric-transformed intensity image is the initial geometric- 
transformed intensity image and the differential data between successive geometric- 
transformed intensity images in the time-series. 

39. An image processing method, comprising: 

projecting light; 

picking up the projected light; 

acquiring an intensity image from the picked-up light; 
acquiring range information from the picked-up light; 

performing geometric transformation for the intensity image based on the range 
information; and 

extracting difference between the geometric-transformed intensity image and thean 
intensity image acquired in advanc e . advance: 

storing, as the geometric-transformed intensity image, an initial geometric- 
transformed intensity image in a time-series transformed in the geometric transformation step: 

making comparison between successive geometric-transformed intensity images in the 

time-series transformed in the geometric transformation step: and 

retrieving only differential data between successive geometric-transformed intensity 

images in the time-series as storage data based on a result of the comparison step of the initial 
geometric-transformed intensity image and geometric-transformed intensity images 
subsequently transformed in the time-series, wherein the stored geometric-transformed 
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intensity image is the initial geometric-transformed intensity image and the differential data 
between successive geometric-transformed intensity images in the time-series. 

40. A storage medium readable by a computer, the storage medium storing a program 
of instructions executable by the computer to perform method steps for performing image 
processing, the method comprising the steps of: 

projecting light; 

picking up the projected light; 

acquiring an intensity image from the picked-up light; 
acquiring range information from the picked-up light; 

performing geometric transformation for the intensity image based on the range 
information; and 

extracting difference between the geometric-transformed intensity image and thean 
intensity image acquired in advanc e . advance; 

storing, as the geometric-transformed intensity image, an initial geometric- 
transformed intensity image in a time-series transformed in the geometric transformation step; 

making comparison between successive geometric-transformed intensity images in the 

time-series transformed in the geometric transformation step; and 

retrieving only differential data between successive geometric-transformed intensity 

images in the time-series as storage data based on a result of the comparison step of the initial 
geometric-transformed intensity image and geometric-transformed intensity images 
subsequently transformed in the time-series, wherein the stored geometric-transformed 
intensity image is the initial geometric-transformed intensity image and the differential data 
between successive geometric-transformed intensity images in the time-series. 
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